Pre-Training on Mixed Data for Low-Resource Neural Machine Translation

The pre-training fine-tuning mode has been shown to be effective for low resource neural machine translation. In this mode, models trained on monolingual data are used initiate translation transfer knowledge from into models. recent years, usually take sentences with randomly masked words as input, and by predicting these based unmasked words. paper, we propose a new method that still predicts words, but replaces some of the in input their another language. bilingual data, so contains both data. We conduct experiments Uyghur-Chinese corpus evaluate our method. experimental results show can make model have better generalization ability help achieve performance. Through word task, also demonstrate enables embedding acquire more alignment knowledge.